• Can be viewed as BLP of \(Y\) or BLP of CEF (\({\textrm E}[Y \mid X]\))
• Solution: \(\beta = \frac{\text{Cov}[X, Y]}{{\textrm V}[X]}\), \(\alpha = {\textrm E}[Y] - \beta {\textrm E}[X]\)

Plug-in estimator: to get \(\hat{\alpha}\), \(\hat{\beta}\), plug in sample covariance, sample variance, sample means to solution above

• Can be viewed as BLP of \(Y\) or BLP of CEF (\({\textrm E}[Y \mid \mathbf{X}]\))
• Solution: no simpler statement than the problem above (except \(\left(\mathbb{X}'\mathbb{X}\right)^{-1} \, \mathbb{X}'\mathbf{Y}\))

Plug-in estimator (Ordinary Least Squares, OLS):

\[(\hat{\beta_0}, \hat{\beta_1}, \ldots, \hat{\beta_K}) = \underset{(b_0, b_1, \ldots, b_K) \in \mathbb{R}^{K+1}}{\arg\min} \, \frac{1}{n}\sum_{i =1}^n\, \left(Y - (b_0 + b_1X_{[1]} + \ldots + b_K X_{[K]})\right)^2,\] i.e. choose coefficients to minimize the mean of the squared residuals in the sample

We could try all possible vectors of coefficients to find \((\hat{\beta_0}, \hat{\beta_1}, \ldots, \hat{\beta_K})\).

The solution (via linear algebra and calculus) is \[\hat{\boldsymbol{\beta}} = \left( \mathbb{X}^T \mathbb{X} \right)^{-1} \mathbb{X}^T \mathbf{Y}\]

where \(\mathbb{X}\) is an \(n \times K+1\) matrix of predictors and \(\mathbf{Y}\) is an \(n \times 1\) column vector (the dependent variable).

We’ll discuss more.

For now we use lm() or estimatr::lm_robust() in R.

Let \(e_i = Y_i - \hat{Y}_i\) be the residual for \(i\) from the model.

Let \(\varepsilon_i = Y_i - \overline{Y}\) be the residual for \(i\) from a null model – no predictors.

“By what proportion does the mean squared residual go down (compared to null model) when we include our predictors?”

\[\begin{align}R^2 &= \frac{\frac{1}{n} \sum_{i = 1}^n \varepsilon_i^2 - \frac{1}{n} \sum_{i = 1}^n e_i^2}{\frac{1}{n} \sum_{i = 1}^n \varepsilon_i^2} \\ &= 1 - \frac{\sum_{i = 1}^n e_i^2}{\sum_{i = 1}^n \varepsilon_i^2}\end{align}\]

• Predict \(Y\): BLP is a predictive model (e.g. infant mortality in a region predicted using data on water quality and electrification)
• Describe relationship between \(X\) and \(Y\) with
  • a BLP coefficient, with or without “controlling for” other factors (e.g. linear relationship between water quality and infant mortality, adjusting for electrification)
  • a function of BLP coefficients and sample features (e.g. average partial derivative of infant mortality with respect to water quality, adjusting for electrification)
• Estimate causal estimands relating to effect of treatment \(D\) on outcome \(Y\) with
  • a regression coefficient (e.g. difference in means in experimental data)
  • predictive models for potential outcomes \(Y(1)\) and \(Y(0)\)

The BLP is linear in the sense that it is the weighted sum of components \(X_{[1]}, X_{[2]}, \ldots, X_{[K]}\):

\[\beta_0 + \beta_1 X_{[1]} + \beta_2 X_{[2]} + \ldots + \beta_K X_{[K]}\]

But the BLP does not need to be a linear function of any particular variable, e.g. GDPPC or education. For example:

• polynomials: \(X_{[1]}\) could be \(\text{GDPPC}\), \(X_{[2]}\) could be \(\text{GDPPC}^2\)
• binning: \(X_{[1]}\) could be 1 if \(\text{GDPPC} \in [1000, 2000)\) and otherwise 0, \(X_{[2]}\) could be 1 if \(\text{GDPPC} \in [2000, 3000)\) and otherwise 0, etc
• interactions: \(X_{[1]}\) could be \(\text{GDPPC}\), \(X_{[2]}\) could be \(\text{area}\), \(X_{[3]}\) could be \(\text{GDPPC} \times \text{area}\)

Key variables:

• treat_deshawn (1 if email from “Deshawn Jackson”, 0 if email from “Jake Mueller”)
• reply_atall (1 if legislator responded, 0 otherwise)
• leg_R (1 if legislator Republican, 0 otherwise)

If we write the prediction equation as \(\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 \mathrm{DeShawn}_i + \hat{\beta}_2 \mathrm{LegRep}_i,\) then we have

So what is the predicted DeShawn-vs-Jake difference for Democrats? For Republicans?

Let’s let \(a_2\) depend (linearly) on the value of \(X_i\)!

\[a_2 = b_0 + b_1 X_i \]

Substitute in \[Y_i = a_0 + a_1 X_i + (b_0 + b_1 X_i) D_i \] and rearrange \[Y_i = a_0 + a_1 X_i + b_0 D_i + b_1 X_i D_i\]

(Same result if we make \(a_1\) depend on \(D_i\).)

If we write the prediction equation as \[\hat{Y}_i = \hat{\beta}_0 + \hat{\beta}_1 \mathrm{DeShawn}_i + \hat{\beta}_2 \mathrm{LegRep}_i + \hat{\beta}_3 \mathrm{DeShawn}_i \times \mathrm{LegRep}_i\], then we have

If we want a CI or \(p\)-value, we need the standard error of a (weighted) sum of coefficients.

Recall variance rule: in general form, \[{\textrm V}[aX + bY + c] = a^2{\textrm V}[X] + b^2{\textrm V}[Y] + 2 a b \text{Cov}[X, Y]\]

Is e.g. \(\text{Cov}[\hat{\beta}_0, \hat{\beta}_1] = 0\) in \(Y_i = \hat{\beta}_0 + \hat{\beta}_1 \text{Deshawn}_i\)?

Standard error of effect of Deshawn email for Republicans:

coef_names <- c("treat_deshawn", "treat_deshawn:leg_R")
(vcov_part <- vcov(intx_reg)[coef_names, coef_names])

                    treat_deshawn treat_deshawn:leg_R
treat_deshawn        0.0003637887       -0.0003637887
treat_deshawn:leg_R -0.0003637887        0.0008145863

(sigma_hat_R <- sqrt(sum(vcov_part))) # V(X) + V(Y) + 2*Cov(X, Y)

[1] 0.021232

Suppose the dots below are the population, and the red line is the CEF:

As we increase \(k\), what will happen to mean squared residual in sample (i.e. \(\frac{1}{n} \sum (Y_i - \hat{Y}_i)^2\)) vs in population (i.e. \(\frac{1}{N} \sum (Y_i - \hat{Y}_i)^2\))?

How does this depend on sample size \(n\)?